Generalization Bounds via Information Density and Conditional Information Density

We present a general approach, based on an exponential inequality, to derive bounds on the generalization error of randomized learning algorithms. Using this approach, we provide bounds on the average generalization error as well as bounds on its tail probability, for both the PAC-Bayesian and single-draw scenarios. Specifically, for the case of subgaussian loss functions, we obtain novel bounds that depend on the information density between the training data and the output hypothesis. When suitably weakened, these bounds recover many of the information-theoretic available bounds in the literature. We also extend the proposed exponential-inequality approach to the setting recently introduced by Steinke and Zakynthinou (2020), where the learning algorithm depends on a randomly selected subset of the available training data. For this setup, we present bounds for bounded loss functions in terms of the conditional information density between the output hypothesis and the random variable determining the subset choice, given all training data. Through our approach, we recover the average generalization bound presented by Steinke and Zakynthinou (2020) and extend it to the PAC-Bayesian and single-draw scenarios. For the single-draw scenario, we also obtain novel bounds in terms of the conditional $\alpha$-mutual information and the conditional maximal leakage.Comment: Published in Journal on Selected Areas in Information Theory (JSAIT). Important note: the proof of the data-dependent bounds provided in the paper contains an error, which is rectified in the following document: https://gdurisi.github.io/files/2021/jsait-correction.pd

New constraints on inelastic dark matter from IceCube

We study the capture and subsequent annihilation of inelastic dark matter (DM) in the Sun, placing constraints on the DM-nucleon scattering cross section from the null result of the IceCube neutrino telescope. We then compare such constraints with exclusion limits on the same cross section that we derive from XENON1T, PICO and CRESST results. We calculate the cross section for inelastic DM-nucleon scattering within an extension of the effective theory of DM-nucleon interactions which applies to the case of inelastic DM models characterised by a mass splitting between the incoming and outgoing DM particle. We find that for values of the mass splitting parameter larger than about 200 keV, neutrino telescopes place limits on the DM-nucleon scattering cross section which are stronger than the ones from current DM direct detection experiments. The exact mass splitting value for which this occurs depends on whether DM thermalises in the Sun or not. This result applies to all DM-nucleon interactions that generate DM-nucleus scattering cross sections which are independent of the nuclear spin, including the "canonical" spin-independent interaction. We explicitly perform our calculations for a DM candidate with mass of 1 TeV, but our conclusions qualitatively also apply to different masses. Furthermore, we find that exclusion limits from IceCube on the coupling constants of this family of spin-independent interactions are more stringent than the ones from a (hypothetical) reanalysis of XENON1T data based on an extended signal region in nuclear recoil energy. Our results should be taken into account in global analyses of inelastic DM models.Comment: 18 pages, 6 figure

Generalization Error Bounds via mmth Central Moments of the Information Density

We present a general approach to deriving bounds on the generalization error of randomized learning algorithms. Our approach can be used to obtain bounds on the average generalization error as well as bounds on its tail probabilities, both for the case in which a new hypothesis is randomly generated every time the algorithm is used - as often assumed in the probably approximately correct (PAC)-Bayesian literature - and in the single-draw case, where the hypothesis is extracted only once. For this last scenario, we present a novel bound that is explicit in the central moments of the information density. The bound reveals that the higher the order of the information density moment that can be controlled, the milder the dependence of the generalization bound on the desired confidence level. Furthermore, we use tools from binary hypothesis testing to derive a second bound, which is explicit in the tail of the information density. This bound confirms that a fast decay of the tail of the information density yields a more favorable dependence of the generalization bound on the confidence level.Comment: ISIT 2020. Corrected Corollary 7 and the discussion in section II-

Evaluated CMI Bounds for Meta Learning: Tightness and Expressiveness

Recent work has established that the conditional mutual information (CMI) framework of Steinke and Zakynthinou (2020) is expressive enough to capture generalization guarantees in terms of algorithmic stability, VC dimension, and related complexity measures for conventional learning (Harutyunyan et al., 2021, Haghifam et al., 2021). Hence, it provides a unified method for establishing generalization bounds. In meta learning, there has so far been a divide between information-theoretic results and results from classical learning theory. In this work, we take a first step toward bridging this divide. Specifically, we present novel generalization bounds for meta learning in terms of the evaluated CMI (e-CMI). To demonstrate the expressiveness of the e-CMI framework, we apply our bounds to a representation learning setting, with $n$ samples from $\hat n$ tasks parameterized by functions of the form $f_i \circ h$. Here, each $f_i \in \mathcal F$ is a task-specific function, and $h \in \mathcal H$ is the shared representation. For this setup, we show that the e-CMI framework yields a bound that scales as $\sqrt{ \mathcal C(\mathcal H)/(n\hat n) + \mathcal C(\mathcal F)/n}$, where $\mathcal C(\cdot)$ denotes a complexity measure of the hypothesis class. This scaling behavior coincides with the one reported in Tripuraneni et al. (2020) using Gaussian complexity.Comment: NeurIPS 202

A New Family of Generalization Bounds Using Samplewise Evaluated CMI

We present a new family of information-theoretic generalization bounds, in which the training loss and the population loss are compared through a jointly convex function. This function is upper-bounded in terms of the disintegrated, samplewise, evaluated conditional mutual information (CMI), an information measure that depends on the losses incurred by the selected hypothesis, rather than on the hypothesis itself, as is common in probably approximately correct (PAC)-Bayesian results. We demonstrate the generality of this framework by recovering and extending previously known information-theoretic bounds. Furthermore, using the evaluated CMI, we derive a samplewise, average version of Seeger's PAC-Bayesian bound, where the convex function is the binary KL divergence. In some scenarios, this novel bound results in a tighter characterization of the population loss of deep neural networks than previous bounds. Finally, we derive high-probability versions of some of these average bounds. We demonstrate the unifying nature of the evaluated CMI bounds by using them to recover average and high-probability generalization bounds for multiclass classification with finite Natarajan dimension.Comment: NeurIPS 202

What makes a design process perceived as complete? : a survey by a design of the Harbor Basin (Hamnbassängen) in Helsingborg

The purpose of this master’s thesis has been to explore and describe how and when the landscape architect perceives its design process as finished. During the course, my friend and I have developed a design proposal for the development of a harbour basin (Hamnbassängen) in Helsingborg. The main focus in the design work has been to investigate the possibilities of developing the site in order to prevent flooding’s due to the rising sea level. By examining my design process, interviewing professional landscape architects and study theory on the subject, I hope to find patterns of how and when the landscape architect feel satisfied with the design process. The aim is also to get a better understanding of what happens mentally during the design process and how to deal with the difficulties that this entails. Initially, the work with the design of Hamnbassängen in Helsingborg is described. The reader will be guided through what conditions the work was based on, how the design process was perceived and finally what thoughts I had after the completed design work. A combination of empirical studies, interviews with three professional landscape architects and literature studies on the topic of design processes, has been the basis for answering the question:” What makes a design process perceived as complete?” Furthermore, the studies have supported me in reflecting upon my own design process in order to get an insight into the factors that made me less satisfied with the design process during the work with Hamnbassängen in Helsingborg. Conclusions that could be drawn about the design process are that the factors that generally affect the perception of feeling complete with the process among others are; Personality, Objectives, Time Frames, Relevant Boundaries, Teamwork, Feedback, and Overall Concept. When these aspects function and interact constructively, the best conditions for perceiving a design process as complete can be found. In general, it seems difficult to feel completely finished with that result and the design process. The feeling of being finished with the work seems to depend on the content of the design process and how you handle different events during the design process

Kritikrätten i Sverige och Danmark - en jämförelse av rätten att kritisera arbetsgivaren länderna emellan

Kritikrätten handlar enligt det arbetsrättsliga synsättet om arbetstagarnas rätt att kritisera sin arbetsgivare. I den här uppsatsen berörs den externa kritikrätten, dvs. rätten att kritisera arbetsgivaren inför utomstående. Sådan kritik är naturligtvis känsligt för arbetsgivaren då den kan tillfoga skada i renomméhänseende. Syftet med uppsatsen är att jämföra kritikrätten i Sverige och Danmark med fokus dels på privat anställda, dels på offentligt anställda, för att se om det finns några skillnader mellan länderna. Anledningen till uppdelningen på sektorer är att de lyder under olika regler. Även om privat anställda har rätt till yttrandefrihet som ges enligt grundlagen och Europakonventionen inträder i och med anställningskontraktet en lojalitetsplikt i vilken det inkluderas en generell tystnadsplikt, en tystnadsplikt som finns i alla anställningsavtal. Denna ska visa sig begränsa yttrandefriheten och därmed rätten att kritisera arbetsgivaren för privat anställda. Enligt lojalitetsplikten är det nämligen förbjudet att skada arbetsgivaren t.ex. genom uttalanden med känsligt innehåll eller genom häftig kritik. Överskrids kritikrätten riskerar arbetstagaren uppsägning eller avskedande. Denna begränsning av kritikrätten som följer av anställningsavtalet finns i både svensk och dansk rätt. För offentligt anställda förhåller det sig annorlunda. För dessa är grundlagen och Europakonventionen fullt tillämplig och offentligt anställdas yttrandefrihet och kritikrätt kan endast begränsas genom lag. Med andra ord blir den tystnadsplikt som följer av lojalitetsplikten utan betydelse för offentligt anställda. Istället finns det regler om tystnadsplikt i lag. Även för offentligt anställda gäller liknande regler för Danmark och Sverige. Således finns samma uppdelning i Danmark där offentligt anställda har väsentligt vidare kritikrätt än privat anställda. Denna brist på yttrandefrihet för privat anställda har kritiserats. I en dom från början av seklet erkänner Europadomstolen att artikel 10 om yttrandefrihet även kan ha betydelse mellan privata rättssubjekt, alltså mellan en privat arbetsgivare och arbetstagare. Beroende på hur domen tolkas skulle det kunna innebära större rättigheter och därmed en vidare kritikrätt för privat anställda men exakt hur domen ska tolkas är i dagsläget oklart

Group assessment challenges in project-based learning – Perceptions from students in higher engineering courses

Industry and society want to recruit students who can work in team-based projects. Thus the task for educators in higher education is to prepare and provide such learning environments. However, assessment is one major challenge associated with enacting these learning environments. The literature advocates active team learning but then supports individual assessment modes. The purpose of this paper is to identify and elaborate on group assessment challenges for students and educators in project-based learning. The research is based on a literature review in the field of project-based learning and group assessment. It is empirically supported by action research in three classes of university engineering students. The findings point to an assessment dilemma, which requires a change in mind-set from individual to team/group grading. The students prefer group learning over written exams. However, when it comes to assessment, the majority want individual grading. Individual assessment is perceived as more fair but unnecessary for learning. Furthermore, a challenge identified by educators is to ensure that all individuals have achieved the learning outcomes. At the same time, they find it frustrating to make individual assessments when the course is based on group learning

CONTRIBUTIONS TO CLUB VELOCITY IN GOLF SWINGS TO SUBMAXIMAL AND MAXIMAL SHOT DISTANCES

The contribution of joint rotations to endpoint velocity was investigated in golf shots to submaximal and maximal shot distances using a 41degrees of freedom (DOF) kinematic model. A subset of 16 DOFs was found to explain 97%-99% of endpoint velocity regulation at club–ball contact. The largest contributors, for both groups at every shot condition, were pelvis and torso twist rotation among the most proximal DOFs, elbow pronation/supination and wrist flexion/extension among DOFs in the left arm, and shoulder internal/external rotation and wrist flexion/extension among DOFs in the right arm. The contributions from pelvis obliquity, left wrist flexion/extension, left wrist ulnar/radial deviation and right shoulder flexion/extension differed significantly between the advanced and intermediate group

Adaptive Selective Sampling for Online Prediction with Experts

We consider online prediction of a binary sequence with expert advice. For this setting, we devise label-efficient forecasting algorithms, which use a selective sampling scheme that enables collecting much fewer labels than standard procedures, while still retaining optimal worst-case regret guarantees. These algorithms are based on exponentially weighted forecasters, suitable for settings with and without a perfect expert. For a scenario where one expert is strictly better than the others in expectation, we show that the label complexity of the label-efficient forecaster scales roughly as the square root of the number of rounds. Finally, we present numerical experiments empirically showing that the normalized regret of the label-efficient forecaster can asymptotically match known minimax rates for pool-based active learning, suggesting it can optimally adapt to benign settings